Generalization Error of Combined Classifiers

Generalization Error of Combined Classifiers 32

Empirical Margin Distributions and Bounding the Generalization Error of Combined Classifiers

We prove new probabilistic upper bounds on generalization error of complex classifiers that are combinations of simple classifiers. Such combinations could be implemented by neural networks or by voting methods of combining the classifiers, such as boosting and bagging. The bounds are in terms of the empirical distribution of the margin of the combined classifier. They are based on the methods ...

Bounding the Generalization Error of Neural Networks and Combined Classifiers

Recently, several authors developed a new approach to bounding the generalization error of complex classi-ers (of large or even innnite VC-dimension) obtained by combining simpler classiiers. The new bounds are in terms of the distributions of the margin of combined classiiers and they provide some theoretical explanation of generalization performance of large neu-We obtained new probabilistic ...

Improved Boosting Algorithm Using Combined Weak Classifiers

From family of corrective boosting algorithms (i.e. AdaBoost, LogitBoost) to total corrective algorithms (i.e. LPBoost, TotalBoost, SoftBoost, ERLPBoost), we analysis these methods of sample weight updating. Corrective boosting algorithms update the sample weight according to the last hypothesis; comparatively, total corrective algorithms update the weight with the best one of all weak classifi...

Direct Optimization of Margins Improves Generalization in Combined Classifiers

Cumulative training margin distributions for AdaBoost versus our "Direct Optimization Of Margins" (DOOM) algorithm. The dark curve is AdaBoost, the light curve is DOOM. DOOM sacrifices significant training error for improved test error (horizontal marks on margin= 0 line)_ -1 -0.8 -0.6 -0.4 -0.2 0 0.2 0.4 0.6 0.8 1 Margin

Generalization error bounds for classifiers trained with interdependent data

In this paper we propose a general framework to study the generalization properties of binary classifiers trained with data which may be dependent, but are deterministically generated upon a sample of independent examples. It provides generalization bounds for binary classification and some cases of ranking problems, and clarifies the relationship between these learning tasks.